The effect of auditory and visual degradation on audiovisual perception of native and non-native speakers

This study investigated the effect of visual and auditory degradation on the weighting given by listeners to visual cues for tokens produced by native and non-native speakers. ‘Ba’, ‘da’ and ‘ga’ syllables uttered by English and Chinese speakers were presented in auditory (A), visual (V) and congruent/incongruent audiovisual (AV) conditions to English listeners. In some conditions, the visual channel was degraded via Gaussian blurring, the auditory channel by pink noise, or both channels were degraded. In A or V conditions, degradation reduced intelligibility. In AV conditions with one degraded channel, the weighting of the other channel increased, with little effect of speaker-language. However, in clear or combined-degradation conditions, listeners gave greater weight to visual cues for non-native speakers. The weighting of auditory and visual cues is therefore highly flexible and there is strong support for a ‘foreign-language effect’ in visual weighting.